In many computerized systems, concurrency may be achieved by sharing the resources of a central processing unit (CPU) between multiple users, multiple programs, multiple parts of a program, multiple events or multiple tasks. In many server applications, such as Web servers, database servers, file servers or mail servers, the number of tasks is large and the processing time of each individual task is short.
Typically, a pool of threads is created to perform the plurality of tasks, which are usually organized in a queue. In many applications, the number of threads remains constant to save thread creation and destruction overhead. The number of threads in the thread pool is chosen to maximize resource usage and performance. The number of threads may be based on percentage usage of the CPU, number of queued requests, and/or the number of processor cores in the system.
In a common implementation of a thread pool, all the tasks needed to be processed are arranged in a single queue and are distributed among the threads according to thread availability. This relatively simple implementation may be problematic when handling several tasks which are related and should be processed in a determined order since in this implementation, the order of processing the tasks may not be controlled.
According to another common solution, referred to as a worker pool, each thread has a dedicated queue referred to as a worker thread, and each task can be assigned to a specific worker thread by a management layer. For example, tasks may be distributed among queues according to a task identification number (ID). In this case, a complex management layer in required in order to ensure optimized processor utilization